The Virtuous Circles of Clinical Information Systems: a Modern Utopia.

CONTEXT: Clinical information systems (CIS) are developed with the aim of improving both the efficiency and the quality of care. OBJECTIVE: This position paper is based on the hypothesis that such vision is partly a utopian view of the emerging eSociety. METHODS: Examples are drawn from 15 years of experience with the fully integrated Georges Pompidou University Hospital (HEGP) CIS and temporal data series extracted from the data warehouses of Assistance Publique - Hôpitaux de Paris (AP-HP) acute care hospitals which share the same administrative organization as HEGP. Three main virtuous circles are considered: user satisfaction vs. system use, system use vs. cost efficiency, and system use vs quality of care. RESULTS: In structural equation models (SEM), the positive bidirectional relationship between user satisfaction and use was only observed in the early HEGP CIS deployment phase (first four years) but disappeared in late post-adoption (≥8 years). From 2009 to 2013, financial efficiency of 20 AP-HP hospitals evaluated with stochastic frontier analysis (SFA) models diminished by 0.5% per year. The lower decrease of efficiency observed between the three hospitals equipped with a more mature CIS and the 17 other hospitals was of the same order of magnitude than the difference observed between pediatric and non-pediatric hospitals. Outcome quality benefits that would bring evidence to the system use vs. quality loop are unlikely to be obtained in a near future since they require integration with population-based outcome measures including mortality, morbidity, and quality of life that may not be easily available. CONCLUSION: Barriers to making the transformation of the utopian part of the CIS virtuous circles happen should be overcome to actually benefit the emerging eSociety.

Validity of a clinical decision rule-based alert system for drug dose adjustment in patients with renal failure intended to improve pharmacists' analysis of medication orders in hospitals.

OBJECTIVE: The main objective of this study was to assess the diagnostic performances of an alert system integrated into the CPOE/EMR system for renally cleared drug dosing control. The generated alerts were compared with the daily routine practice of pharmacists as part of the analysis of medication orders. MATERIALS AND METHODS: The pharmacists performed their analysis of medication orders as usual and were not aware of the alert system interventions that were not displayed for the purpose of the study neither to the physician nor to the pharmacist but kept with associate recommendations in a log file. A senior pharmacist analyzed the results of medication order analysis with and without the alert system. The unit of analysis was the drug prescription line. The primary study endpoints were the detection of drug dose prescription errors and inter-rater reliability (Kappa coefficient) between the alert system and the pharmacists in the detection of drug dose error. RESULTS: The alert system fired alerts in 8.41% (421/5006) of cases: 5.65% (283/5006) "exceeds max daily dose" alerts and 2.76% (138/5006) "under-dose" alerts. The alert system and the pharmacists showed a relatively poor concordance: 0.106 (CI 95% [0.068-0.144]). According to the senior pharmacist review, the alert system fired more appropriate alerts than pharmacists, and made fewer errors than pharmacists in analyzing drug dose prescriptions: 143 for the alert system and 261 for the pharmacists. Unlike the alert system, most diagnostic errors made by the pharmacists were 'false negatives'. The pharmacists were not able to analyze a significant number (2097; 25.42%) of drug prescription lines because understaffing. CONCLUSION: This study strongly suggests that an alert system would be complementary to the pharmacists' activity and contribute to drug prescription safety.

A business rules design framework for a pharmaceutical validation and alert system.

OBJECTIVES: Several alert systems have been developed to improve the patient safety aspects of clinical information systems (CIS). Most studies have focused on the evaluation of these systems, with little information provided about the methodology leading to system implementation. We propose here an 'agile' business rule design framework (BRDF) supporting both the design of alerts for the validation of drug prescriptions and the incorporation of the end user into the design process. METHODS: We analyzed the unified process (UP) design life cycle and defined the activities, subactivities, actors and UML artifacts that could be used to enhance the agility of the proposed framework. We then applied the proposed framework to two different sets of data in the context of the Georges Pompidou University Hospital (HEGP) CIS. RESULTS: We introduced two new subactivities into UP: business rule specification and business rule instantiation activity. The pharmacist made an effective contribution to five of the eight BRDF design activities. Validation of the two new subactivities was effected in the context of drug dosage adaption to the patients' clinical and biological contexts. Pilot experiment shows that business rules modeled with BRDF and implemented as an alert system triggered an alert for 5824 of the 71,413 prescriptions considered (8.16%). CONCLUSION: A business rule design framework approach meets one of the strategic objectives for decision support design by taking into account three important criteria posing a particular challenge to system designers: 1) business processes, 2) knowledge modeling of the context of application, and 3) the agility of the various design steps.

Electronic Healthcare Record and clinical research in cardiovascular radiology. HL7 CDA and CDISC ODM interoperability.

Integrating clinical research data entry with patient care data entry is a challenging issue. At the G. Pompidou European Hospital (HEGP), cardiovascular radiology reports are captured twice, first in the Electronic Health Record (EHR) and then in a national clinical research server. Informatics standards are different for EHR (HL7 CDA) and clinical research (CDISC ODM). The objective of this work is to feed both the EHR and a Clinical Research Data Management System (CDMS) from a single multipurpose form. We adopted and compared two approaches. First approach consists in implementing the single "care-research" form within the EHR and aligning XML structures of HL7 CDA document and CDISC ODM message to export relevant data from EHR to CDMS. Second approach consists in displaying a single "care-research" XForms form within the EHR and generating both HL7 CDA document and CDISC message to feed both EHR and CDMS. The solution based on XForms avoids overloading both EHR and CDMS with irrelevant information. Beyond syntactic interoperability, a perspective is to address the issue of semantic interoperability between both domains.

François Grémy and the birth of IMIA. 1st IMIA/UMIT Medical Informatics Award of Excellence given to Professor Grémy.

In 2001 the International Medical Informatics Association (IMIA) approved the establishment of a Medical Informatics Award of Excellence to be given every three years to an individual, whose personal commitment and dedication to medical informatics has made a lasting contribution to medicine and healthcare through her or his achievements in research, education, development or applications in the field of medical informatics. The first award was given in 2004 to Prof. François Grémy, Uzes, France. As the first chairman and moderator of TC4, François Grémy is considered to be the first President of its renamed and refocused successor, the International Medical Informatics Association. The role of IFIP-TC4 in bringing together early health informaticians cannot be underestimated. Although TC4 was composed in large part of computer professionals interested in medical applications, Grémy recruited the first generation of IMIA officers and members from the medical and healthcare communities. Intellectually as well as organizationally, IFIP-TC4 was the true predecessor of IMIA.

Component-based mediation services for the integration of medical applications.

Allowing exchange of information and cooperation among network-wide distributed and heterogeneous applications is a major need of current health-care information systems. The European project SynEx aims at developing an integration platform for both new and legacy applications on each partner's site. We developed, in this project, mediation services based on the generic and reusable software components that facilitate the construction of an integration platform and ease the communication and the meaningful transformation among distributed and heterogeneous applications. The main component of the mediation services is named Pilot, which serves as an intelligent broker. It uses a multi-agents service model allowing the integration platform to be multi-servers. It transforms a client request into a valid high level service on the platform. Each service is broken up into several elementary steps by the Pilot. For each step, the Pilot uses an agent to realize the operation configured by the step. At runtime, the Pilot synchronizes the execution of different steps. To ease the communication and the interaction with the heterogeneous systems, an agent can integrate a Mediator. The Mediators are the communication and interpretation tools within the mediation services. We have developed a generic model that can be specialized for creating specific mediators for the different use cases. The mediator model uses two interfaces to connect the mediator with two systems that need to communicate. Each interface deals with the three aspects through three managers (the Communication Manager, the Syntax Manager and the Semantic Manager). Some ready-to-use specializations are developed for some well defined cases which can reduce the development effort. Once a manager is specialized, it can be used in different combinations with other managers to resolve different problems. The meaningful transformation is ensured on a semantic level in each mediator through the Semantic Model component. This last component allows the mapping among different vocabularies used by different systems through a shared ontology which allows the mapping process to focus on the meaning of the transformed information. We have used XML in different components of the mediation services as the interchange format and the description format. This has enhanced the flexibility of the components. The component based approach allows the generic components to be reused in different contexts and also allows the mediations services to be open to integrate other available technologies thus largely reduce the development efforts.

Integrating medical applications in an open architecture through generic and reusable components.

Allowing exchange of information and cooperation among network-wide distributed and heterogeneous applications is a major need of current health care information systems. It forces the development of open and modular integration architectures. Major issues in the development include defining a flexible and robust federation model, developing interaction and communication facilities as well as the mechanism insuring semantic interoperability. We developed generic and reusable software components to ease the construction of any integration platform. The Pilot and the Mediator Service components facilitate the execution of services and the meaningful transformation of information. They have been tested in the context of the SynEx European project to construct a multi-agents based integration architecture. The possibility of such architectures to take into account the issue of semantic interoperability is further discussed.

Expression and meaning of medical language: building an epistemological framework for the study of semantic distance.

The use of language for communication purposes introduces a semantic gap in data processing. We have developed a new tool called semantic distance to overcome this gap. A previous presentation of our tool was based on an empirical approach. Here, we examine the properties and theoretical foundation of semantic distance from a rationalistic perspective. We present an epistemological framework to explain the meaning of semantic distance. As a tool, the purpose of semantic distance is the exchange of data or knowledge based on linguistic expressions between software components. We propose a review of the relationship between language and its representation for data processing based on the underlying philosophical assumptions. Description logic is a new paradigm in the medical informatics community to express relations between terms in "is-a" hierarchies. The utility of semantic distance is compared to description logic for communication purposes in different use cases.

Computerization of guidelines: towards a "guideline markup language".

Medical decision making is one of the most difficult daily tasks for physicians. Guidelines have been designed to reduce variance between physicians in daily practice, to improve patient outcomes and to control costs. In fact, few physicians use guidelines in daily practice. A way to ease the use of guidelines is to implement computerised guidelines (computer reminders). We present in this paper a method of computerising guidelines. Our objectives were: 1) to propose a generic model that can be instantiated for any specific guidelines; 2) to use eXtensible Markup Language (XML) as a guideline representation language to instantiate the generic model for a specific guideline. Our model is an object representation of a clinical algorithm, it has been validated by running two different guidelines issued by a French official Agency. In spite of some limitations, we found that this model is expressive enough to represent complex guidelines devoted to diabetes and hypertension management. We conclude that XML can be used as a description format to structure guidelines and as an interface between paper-based guidelines and computer applications.

From paper-based to electronic guidelines: application to French guidelines.

In order to develop an internet-based decision support system, making available for French general practitioners several prevention guidelines is was necessary to implement paper based guideline. We propose a framework allowing to transform paper based practice guideline into their electronic form. Three different problems were identified: computability (e.g. determinism of the eCPG), logic (e.g. ambiguities when combining Booleans operators) and external validity (i.e. stability of decision for variations around thresholds and proportion of subjects classified in the various terminal nodes). The last problem concerned documentation of evidence: the level of evidence was associated only with the terminal decision node and not with the pathway through the decision tree. We concluded that computerisation of guidelines is not possible without expertise or authors advice. To improve computability it is necessary to provide authors with a framework that checks ambiguities, and logical errors.

Guideline development: focus on breast cancer screening in the EsPeR project.

We describe in this paper, the implementation of a clinical practice guideline focused on breast cancer screening. Our aim in conceiving such a computerized guideline was first to help general practitioners in appreciating the risks their female patients might develop breast cancer and secondly to suggest them the screening measures adapted to each particular case. This implementation enables us to present our general methodology to elaborated and promulgate guidelines within the EsPeR project. This methodology aims at providing guidelines based on knowledge validated according to the EBM principles, that can be used in real time and updated according to current knowledge.

Logistic regression model: an assessment of variability of predictions.

Risk prediction models available for cardiovascular prevention are statistical or based on machine learning methods. This paper investigates whether the logistic regression method can be considered as reference for validation of other methods. In order to test the stability of the predictions using this method, we performed two types of analyses on 50 random training and test samples drawn from the same database. In first analyses three models were obtained by forced entry of different sets of four variables. In second analyses, models were built with increasing number of predictive variables. The predictive performance was assessed by the area under the ROC curve. Although across-samples variability is low for a given model, it is large enough to lead to wrong conclusions when comparing different prediction methods. We also suggest that a low events-per-variable ratio alters the stability of a model's coefficients but does not affect the variability of prediction performance.

Elaboration and formalization of current scientific knowledge of risks and preventive measures illustrated by colorectal cancer.

OBJECTIVES: Present the method used to elaborate and formalize current scientific knowledge to provide physicians with tools available on the Internet, that enable them to evaluate individual patient risk, give personalized preventive recommendations or early screening measures. METHODS: The approach suggested in this article is in line with medical procedures based on levels of evidence (Evidence-based Medicine). A cyclical process for developing recommendations allows us to quickly incorporate current scientific information. At each phase, the analysis is reevaluated by experts in the field collaborating on the project. The information is formalized through the use of levels of evidence and grades of recommendations. GLIF model is used to implement recommendations for clinical practice guidelines. RESULTS: The most current scientific evidence incorporated in a cyclical process includes several steps: critical analysis according to the Evidence-based Medicine method; identification of predictive factors; setting-up risk levels; identification of prevention measures; elaboration of personalized recommendation. The information technology implementation of the clinical practice guideline enables physicians to quickly obtain personalized information for their patients. Cases of colorectal prevention illustrate our approach. CONCLUSIONS: Integration of current scientific knowledge is an important process. The delay between the moment new information arrives and the moment the practitioner applies it, is thus reduced.

Identification of patients at high cardiovascular risk: a critical appraisal of applicability of statistical risk prediction models.

Assessment of cardiovascular risk is widely proposed as a basis for taking management decisions about patients presenting with hypertension or hypercholesterolemia. Our aim was to critically assess the use of risk equations derived from epidemiological studies for the purpose of identifying high-risk patients. Risk equations were retrieved from the MEDLINE database and then applied to a data set of 118 patients. This data set was an evaluation study of the clinical value of the World Health Organization 1993 hypertension guidelines for the decision to treat mild hypertensive patients. We calculated agreement: 1) between equations and 2) between equations and the decision to treat taken by the physician. Most models were not applicable to our population, mainly because the original population had a narrow age range or comprised only males. Between-model agreement was better for the lower and upper risk quintiles than for the three other risk quintiles (0.58, 0.33, 0.34, 0.45, 0.70, from the lower to the upper risk quintile). When using an arbitrary threshold for defining high-risk patients (i.e. > 2% per year), we observed a huge variation of the proportion of patients classified at high risk (from 0 to 17%). There was a poor agreement between risk models and the decision to treat taken by the physician. These results suggest that risk-based guidelines should be validated before their diffusion.

Modeling uncertainty in computerized guidelines using fuzzy logic.

Computerized Clinical Practice Guidelines (CPGs) improve quality of care by assisting physicians in their decision making. A number of problems emerges since patients with close characteristics are given contradictory recommendations. In this article, we propose to use fuzzy logic to model uncertainty due to the use of thresholds in CPGs. A fuzzy classification procedure has been developed that provides for each message of the CPG, a strength of recommendation that rates the appropriateness of the recommendation for the patient under consideration. This work is done in the context of a CPG for the diagnosis and the management of hypertension, published in 1997 by the French agency ANAES. A population of 82 patients with mild to moderate hypertension was selected and the results of the classification system were compared to whose given by a classical decision tree. Observed agreement is 86.6% and the variability of recommendations for patients with close characteristics is reduced.

A consensus approach to maintain a knowledge based system in pathology.

The IDEM (Images and Diagnosis from Example in Medicine) software is a computerized environment able to store unambiguous descriptions of histopathologic images from pathologists. Medical imaging could benefit from such environments if they can easily and continuously be maintained. Within the IDEM environment, we developed a knowledge management module coupled with a consensus module to support knowledge acquisition and maintenance by the experts of the domain. Two pathologists, a senior and junior pathologist, reviewed fifty-three cases of breast pathology. Our findings indicate 1) that the IDEM knowledge management module allows experts to describe images by selecting terms and defining new ones if necessary, allowing the construction of a glossary for the domain and 2) that the consensus module, developed to store valid multi-experts cases, contributes also to validate new terms of the glossary and to refine semantic distance between terms. Such methodology could be applied to others highly evolving medical domains.

Using semantic distance for the efficient coding of medical concepts.

OBJECTIVE: To use the notion of semantic distance to find the nearest neighbors of a medical concept in a controlled vocabulary. MATERIAL AND METHOD: 392 concepts from the cardiovascular chapter of the ICD-10 were projected on the axes of SNOMED III. Distances were measured on each axis and the resulting distance was found using a Lp norm. RESULTS: The distance between a set of ischemic diseases and a set of non-ischemic diseases was significant (p < 0.0001). Our method was validated by finding the k nearest neighbors of ten different diagnoses from the ICD-10 cardiovascular chapter. DISCUSSION: The availability of SNOMED-RT should improve our method. Several more steps are necessary to provide an ideal coding tool.

Models to predict cardiovascular risk: comparison of CART, multilayer perceptron and logistic regression.

The estimate of a multivariate risk is now required in guidelines for cardiovascular prevention. Limitations of existing statistical risk models lead to explore machine-learning methods. This study evaluates the implementation and performance of a decision tree (CART) and a multilayer perceptron (MLP) to predict cardiovascular risk from real data. The study population was randomly splitted in a learning set (n = 10,296) and a test set (n = 5,148). CART and the MLP were implemented at their best performance on the learning set and applied on the test set and compared to a logistic model. Implementation, explicative and discriminative performance criteria are considered, based on ROC analysis. Areas under ROC curves and their 95% confidence interval are 0.78 (0.75-0.81), 0.78 (0.75-0.80) and 0.76 (0.73-0.79) respectively for logistic regression, MLP and CART. Given their implementation and explicative characteristics, these methods can complement existing statistical models and contribute to the interpretation of risk.